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EXECUTIVE  SUMMARY 


This  report  discusses  the  results  of  a  preliminary  investigation  of  cues  (e.g.,  speed,  range,  etc.) 
that  experienced  surface  warfare  personnel  use  to  assess  the  threat  level  of  nearby  surface  ships.  Data 
were  collected  from  experienced  U.S.  Navy  surface  warfare  watch  standers  within  a  Combat 
Information  Center  (CIC)  on  an  Aegis-equipped  ship.  The  data  were  incorporated  into  an  outline  of 
an  algorithm  for  surface  threat  assessment.  The  objectives  of  this  investigation  were  as  follows: 

•  Find  the  level  of  threat  associated  with  different  types  of  ships 

•  Describe  the  relationship  between  specific  values  of  cues  and  the  corresponding  perception 
of  threat 

•  Rank  cues  in  order  of  importance  or  relevance  to  threat  assessment 

•  Find  the  flow  of  information  in  CIC 

•  Develop  an  algorithm  for  preliminary  surface  threat  assessment 

RESEARCH  BACKGROUND 

The  data  collection  and  algorithm  development  followed  the  same  process  that  Liebhaber  and 
Smith  (2000)  and  Liebhaber,  Kobus,  and  Smith  (2000)  used  to  provide  theoretical  and  applied  basis 
for  air  threat  assessment.  In  these  studies,  participants  described  a  list  of  1 8  cues  (aircraft  character¬ 
istics)  relevant  to  the  threat  assessment  process.  This  report  discusses  a  similar  list  developed  for 
evaluating  surface  threats.  This  work  developed  from  research  on  Decision  Support  Systems  (DSSs) 
in  the  Tactical  Decision-Making  Under  Stress  (TADMUS)  program.  DSS  supports  the  cognitive 
strategies  of  tactical  decision-makers  operating  in  highly  complex,  fast-paced  littoral  environments. 
While  the  DSS  was  successful,  theoretical  and  applied  investigation  of  the  threat  assessment 
concepts  was  minimal.  This  report  discusses  the  initial  efforts  toward  understanding  the  surface 
warfare  aspects  of  the  threat  assessment  process. 

FINDINGS 

Nine  surface-warfare-experienced  participants  provided  baseline  threat  ratings  for  ship  categories 
(e.g.,  Friend,  Combatant,  Oiler,  etc.)  and  the  relationship  between  perceived  threat  rating  and 
specific  cues  (e.g.,  speed  of  10  to  15  knots  =  raise  threat  rating  a  little).  Baseline  threat  ratings  were 
consistent  with  the  findings  from  air  defense  studies  (Liebhaber  and  Smith,  2000;  Liebhaber,  Kobus, 
and  Smith,  2000).  Participants  also  ranked  the  importance  or  relevance  of  each  assessment  cue  and 
described  the  flow  of  surface  warfare  information.  The  top  three  cues,  tied  for  number  one,  were 
Platform  Type,  Weapon  Envelope,  and  Electronic  Emissions.  Flow  data  provided  an  understanding 
of  the  information  requirements  and  functional  relationships  within  CIC. 


SURFACE  THREAT  ALGORITHM 


Data  were  incorporated  into  a  rule-based,  surface  threat  algorithm.  The  air  threat  algorithm 
(Liebhaber,  2001)  provided  a  procedural  framework  for  the  surface  algorithm.  The  algorithm 
analyzes  relevant  data  and  computes  a  threat  rating  for  a  surface  track. 

RECOMMENDATIONS 

Surface  warfare  data  were  collected  from  highly  experienced  U.S.  Navy  personnel.  The  data 
provide  a  foundation  for  the  development  of  a  working  algorithm.  However,  the  next  phase  of  inves¬ 
tigation  should  include  an  empirical  determination  of  cue  use  patterns  and  cue  weights.  Those  find¬ 
ings  can  then  be  incorporated  into  future  development  of  the  algorithm  for  surface  warfare  threat 
assessment  and  interface  guidelines. 


This  is  a  work  of  the  United  States  Government  and  therefore  is  not  copyrighted.  This  work  may 
be  copied  and  disseminated  without  restriction.  Many  SSC  San  Diego  public  release  documents  are 
available  in  electronic  format  at  http://www.spawar.navy.mil/sti/publications/pubs/index.html. 
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1.  INTRODUCTION 


The  Surface  Warfare  Threat  Assessment  project  had  two  major  goals: 

1 .  Identify  information  that  experienced  surface  warfare  (SUW)  personnel  use  in  assessing  the 
threat  level  of  nearby  surface  craft. 

2.  Develop  an  algorithm  for  preliminary  surface  threat  assessment. 

To  reach  these  goals,  a  knowledge  engineering  study  identified  the  specific  assessment  cues  that 
individuals  and  expert  tactical  decision-makers  use  on  a  ship’s  Combat  Information  Center  (CIC). 
Data  were  also  collected  on  the  relationship  between  the  perceived  threat  rating  of  surface  ships  and 
the  specific  cues  (e.g.,  speed  over  15  knots).  This  report  discusses  study  results  and  outlines  a 
preliminary  surface  threat  algorithm. 

Surface  ship  threat  assessment  involves  evaluating  cues  from  tracks  (radar  contacts)  to  identify 
and  assess  threats  to  the  ship  or  battle  group.  Cues  (also  called  data,  features,  or  attributes)  are  char¬ 
acteristics  associated  with  tracks  such  as  origin,  speed,  and  range.  No  one  system  or  person  on  the 
ship  provides  conclusive  identification  or  threat  assessment.  An  interaction  between  sensor  systems, 
computers,  and  human  operators  within  the  CIC  produces  these  tasks.  The  tasks  need  a  team  of 
highly  trained  people  to  evaluate,  integrate,  and  judge  information. 

When  warfighters  have  a  poor  understanding  of  the  tactical  situation,  they  may  rank  threats  inac¬ 
curately  and  share  tactical  resources  inadequately.  Current  systems  produce  tactical  pictures  that  can 
be  incomplete,  misleading,  hard  to  interpret,  and  untimely.  Adequate  decision  support  systems 
(DSSs)  must  allow  warfighters  to  manage  information  quickly,  tailor  it  for  mission  requirements,  and 
help  them  tactically  grasp  mission-critical  information.  The  research  reported  here  is  one  part  of  a 
research  program  that  will  meet  those  requirements.  This  program  focused  on  the  following  goals: 

1 .  Determine  the  information  that  expert  tactical  decision-makers  use  in  evaluating  potential 
threat  tracks  in  ambiguous  operational  situations 

2.  Build  an  executable  model  of  threat  assessment  for  a  decision-support  tool 

3.  Validate  the  decision-support  tool(s)  in  operation 

This  effort  will  produce  tools  that  help  tactical  warfighters  assess  contacts,  rank  contacts  in  their 
degree  of  threat,  and  develop  appropriate  courses  of  action  to  reduce  threat  uncertainty,  which  will 
increase  tactical  awareness. 

The  specific  objectives  of  this  study  were  as  follows: 

1 .  Find  the  level  of  threat  associated  with  different  types  of  ships 

2.  Describe  the  relationship  between  specific  values  of  cues  and  the  corresponding  perception 
of  threat 

3.  Rank  cues  in  order  of  importance  or  relevance  to  threat  assessment 
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4.  Find  the  flow  of  information  in  CIC 


5.  Develop  an  algorithm  for  preliminary  surface  threat  assessment 

Data  collection  and  development  followed  the  same  process  used  for  developing  an  air  defense 
algorithm  (Liebhaber,  2001).  Liebhaber  and  Smith  (2000)  and  Liebhaber,  Kobus,  and  Smith  (2000) 
provided  a  cognitive  basis  for  the  algorithm.  They  investigated  the  practice  of  threat  assessment 
within  U.S.  Navy  single-ship  air  defense.  The  studies  provided  answers  to  questions  such  as  the 
following:  What  information  do  air  defense  personnel  use  to  assess  aircraft  contacts  and  do  they  use 
it  systematically?  In  these  studies,  participants  listed  18  cues  (aircraft  characteristics)  relevant  to  the 
threat  assessment  process.  A  similar  list  was  developed  for  evaluating  surface  threats  from  the  results 
of  this  study. 

This  work  developed  from  research  on  DSSs  in  the  Tactical  Decision-Making  Under  Stress 
(TADMUS)  program.  TADMUS  focused  on  evaluating  DSS  display  concepts  derived  from  cognitive 
theory,  most  notably,  Naturalistic  Decision-Making  (Zsambok  and  Klein,  1997).  DSS  supports  the 
cognitive  strategies  of  tactical  decision-makers  operating  in  highly  complex,  fast-paced  littoral 
environments.  See  Hutchins  (1996),  Rummel  (1995),  and  Morrison  (2000)  for  reviews  of  DSS.  One 
TADMUS  DSS  component  was  called  the  Basis  for  Assessment  (BFA)  tool.  The  tool  was  a  display 
section  with  explanation-based  threat  assessment.  System  operators  could  view  a  detailed  list  of 
evidence  for  and  against  the  current  assessment  of  the  selected  track.  Pennington  and  Hastie  (1988) 
described  a  similar  display.  The  tool  reduces  the  likelihood  of  mis-categorizing  and  engaging 
friendly  or  neutral  tracks  (Morrison  et  ah,  1997).  DSS  was  successful,  but  the  threat  assessment 
concepts  were  investigated  further.  Liebhaber  and  Smith  (2000)  and  Liebhaber,  Kobus,  and  Smith 
(2000)  initiated  studies  to  better  understand  the  threat  assessment  process  in  air  warfare  and  to 
develop  better  algorithms  and  displays.  This  report  discusses  the  initial  efforts  toward  understanding 
the  threat  assessment  process  for  surface  warfare. 
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2.  METHOD 


A  questionnaire  gathered  data  on  the  information  that  SUW  personnel  use  to  assess  the  threat 
level  of  surface  craft  within  their  ship’s  surveillance  area.  Appendix  A  includes  the  questionnaire. 

2.1  PARTICIPANTS 

Nine  U.S.  Navy  personnel  with  at-sea  experience  in  SUW  participated  in  this  study.  They  had  a 
mean  of  7.9  years  (SD  =  4.8  years;  Range  =  3.7  to  12.1  years)  of  at-sea  experience  in  CIC.  They 
were  experienced  in  an  average  of  3.3  (SD  =  1.3)  CIC  watch  stations.  Their  experience  included  an 
average  of  3.4  years  as  the  SUW  Coordinator,  the  focal  point  of  Anti-Surface  Warfare  in  CIC. 
Appendix  B  describes  the  participants’  CIC  watch  station  experience. 

2.2  QUESTIONNAIRE 

Items  on  the  questionnaire  were  developed  from  U.S.  Navy  documents  and  with  current  and 
former  SUW  personnel.  The  questionnaire’s  first  page  gathered  data  regarding  each  participant’s 
SUW  experience.  The  rest  of  the  questionnaire  was  divided  into  four  parts. 

Parti 

The  first  set  of  questions  identified  baseline  threat  levels  for  Flag  (ID  or  Origin;  indicates  the 
country  from  which  the  track  most  likely  originated)  classifications  (e.g..  Unknown,  Neutral,  Enemy, 
etc.)  and  different  type  of  ships  (e.g.,  Combatant,  Patrol,  Cargo,  etc.).  Threat  was  defined  as  the 
perceived  ability  of  a  track  to  inflict  damage  on  one’s  own  ship  or  battle  group.  It  pertained  only  to 
the  degree  of  the  threat  posed  by  a  particular  track.  Participants  were  asked  to  respond  for  two 
operational  environments:  littoral  (near  shore)  and  open  ocean.  Earlier  work  (Liebhaber  and  Smith, 
2000)  found  differences  in  threat  levels  for  each  environment.  Participants  responded  on  a  5 -point 
Likert  scale  (range  from  1  =  Never  Threatening  to  5  =  Always  Threatening).  Table  1  is  an  example 
for  the  question:  “How  threatening,  on  average,  is  this  type  of  ship?  (Please  circle  your  response)”. 


Table  1.  Threat  level  response  example. 


Military  Ships 

Never 

Rarely 

Sometimes 

Often 

Always 

Carrier 

1 

2 

3 

4 

5 

Surface 

Combatant 

1 

2 

3 

4 

5 

Patrol/Escort 

1 

2 

3 

4 

5 

Oiler 

1 

2 

3 

4 

5 

Sealift 

1 

2 

3 

4 

5 

3 


Part  2 


Participants  estimated  change  to  baseline  threat  level  for  15  cues.  Appendix  C  lists  the  cues  with 
definitions.  Each  cue  lists  data  values,  as  shown  below  for  Range.  Transcripts  of  verbal  protocols  in 
Liebhaber  and  Smith  (2000)  and  from  SUW  subject  matter  experts  determined  the  data  values. 
Participants  were  instructed  to  treat  each  cue  (e.g..  Range)  as  the  first  piece  of  data  received,  and 
then  to  indicate  how  that  data  would  change  their  estimate  of  threat.  They  were  told  to  answer  based 
on  their  at-sea  experience.  All  questions  were  about  single  pieces  of  information  (e.g.,  speed,  range, 
heading,  etc.).  Although  information  is  not  processed  in  isolation  in  the  real  situation,  our  goal  was 
to  understand  how  participants  evaluated  individual  elements.  Participants  responded  on  a  scale  from 
1  (lower  threat  level  greatly)  to  5  (raise  threat  level  greatly).  Table  2  shows  an  example  of  data  and 
response  values  for  Range  are  shown  below. 


Table  2.  Range  data  and  response  values. 


Range 

Lower 

Greatly 

Lower  a 
Little 

No 

Change 

Raise  a  Little 

Raise 

Greatly 

Under  5  nmi 

1 

2 

3 

4 

5 

5  to  25  nmi 

1 

2 

3 

4 

5 

25  to  50nmi 

1 

2 

3 

4 

5 

Over  50  nmi 

1 

2 

3 

4 

5 

Part  3 

Participants  ranked  the  importance  or  relevance  of  1 4  assessment  cues  (Cargo  was  inadvertently 
omitted)  and  three  other  items:  Origin,  Superstructure  Type,  and  Platform  Type.  They  ranked  17 
items.  The  ranking  was  from  1  (item  most  relevant  to  determining  threat)  to  17  (item  least  relevant  to 
determining  threat).  The  rankings  were  a  rough  index  to  each  cue’s  relative  weight.  Relative  weights 
are  used  in  the  assessment  algorithm  and  are  derived  empirically  (see  Liebhaber,  Kobus.  and  Smith, 
2000). 

Part  4 

The  final  questions  asked  participants  to  identify  from  whom  they  received  information,  and  to 
whom  they  passed  information.  The  purpose  was  to  better  understand  the  flow  of  data  and  the 
functional  relationships  within  CIC.  Flow  data  are  important  for  identifying  and  understanding  the 
process  of  threat  assessment  and  course  of  action  selection.  The  data  show  the  communication 
pathways  used  most  often  and  highlight  potential  areas  where  enhanced  displays  may  help  reduce 
communication  gaps  and  task  interruptions. 

2.3  PROCEDURE 

Each  participant  was  briefed  about  the  study’s  purpose.  All  participants  then  filled  out  a  written 
questionnaire  that  included  questions  about  their  military  experience  and  track  threat  ratings.  The 
questionnaire  took  about  15  minutes  to  finish. 
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3.  RESULTS 


The  baseline  threat  levels  for  categories  of  tracks  are  reported  first,  followed  by  the  relationship 
between  cue  data  and  corresponding  perception  of  threat,  and  then  cue  rankings  and  information 
flow. 

3.1  BASELINE  THREAT  RATINGS  FOR  SHIPS 

Part  1  of  the  questionnaire  asked  participants  to  rate  the  perceived  threat  of  tracks  in  standard 
categories  (e.g.,  Friend,  Enemy,  etc.).  Table  1  shows  mean  ratings.  Participants  responded  to  all 
questions  in  Part  1,  but  Table  1  shows  only  selected  categories.  Threat  ratings  in  the  littoral  envi¬ 
ronment  tended  to  be  higher  than  in  the  open  ocean  environment.  This  finding  was  consistent  with 
data  from  the  air  threat  assessment  studies  (Liebhaber,  Kobus,  and  Smith,  2000).  The  values  in  Table 
3  are  the  baseline  or  default  threat  levels  for  surface  tracks  in  these  categories. 


Table  3.  Mean  threat  rating  in  each  operational  environment. 


Littoral 

Open  Ocean 

Type  or  Category 

Mean 

Stdev 

Mean 

Stdev 

Flag/Origin 

Friend 

2.11 

1.27 

1.78 

0.97 

Unknown 

2.89 

1.36 

2.89 

1.27 

Hostile 

4.22 

0.83 

4.00 

0.87 

Hull  Type 

Type  1 

2.63 

1.69 

2.14 

1.46 

Military  Ship 

Carrier 

3.33 

1.32 

3.33 

1.50 

Patrol/Escort 

3.67 

0.87 

3.44 

1.01 

Service  Craft 

2.00 

1.22 

1.67 

0.71 

Other  Auxiliary 

2.00 

1.22 

1.67 

0.71 

Commercial/Private  Ship 

Sealift 

1.89 

1.27 

1.56 

0.73 

Fishing 

1.89 

1.27 

1.56 

0.73 

3.2  PERCEPTION  OF  THREAT 

Participants  estimated  changes  to  the  baseline  threat  level  for  15  cues.  An  example  for  Range  was 
shown  above.  Participants  responded  on  a  scale  from  1  (lower  threat  level  greatly)  to  5  (raise  threat 
level  greatly).  Their  mean  responses  were  converted  to  threat  level  change  ratings  (TCRs).  TCRs 
ranged  from  -2  (lower  threat  level  greatly)  to  0  (no  change  to  threat  level)  to  +2  (raise  threat  level 
greatly).  The  ratings  were  converted  to  index  the  magnitude  of  change  and  the  rise  or  fall  in  baseline 
threat  level.  They  are  also  easily  incorporated  into  the  threat  assessment  algorithm.  Participants 
responded  to  all  questions  on  the  questionnaire;  however,  Table  4  shows  only  selected  TCRs  for 
each  cue  in  each  environment.  As  an  example,  a  ship  that  is  slowing  down  in  a  littoral  environment 
will  increase  in  threat  rating  by  0.44. 
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Table  4.  Selected  TCRs  in  each  operational  environment. 


Cues  with  Data  Values 

Littoral 

Ocean 

Speed 

Steady 

-0.22 

-0.22 

Decrease 

0.44 

0.33 

Heading 

Steady/Closing 

1.44 

1.33 

CPA 

Under  1  nmi 

1.78 

1.78 

Recent  history 

On  standard  patrol/route 

0.50 

0.44 

Unknown 

0.56 

0.67 

Range 

Under  5  nmi 

1.56 

1.44 

Over  50  nmi 

-1.00 

-0.89 

Cargo 

Possible  contraband 

0.89 

0.89 

Number  of  vessels 

Single  contact 

0.11 

0.11 

Sea  Lane 

On/Following 

-0.11 

0.00 

Emission  Sensing  Monitor  (ESM) 
No  Emitter 

0.67 

0.67 

Coordinated  Activity 
Training/Exercise 

0.33 

0.33 

Voice  Communication  with  Track 
Have  Communication  with 
Track 

0.67 

0.67 

Own  Support  in  Area 

DCA  or  Helicopter  Available 

0.56 

0.56 

Destination 

Unknown 

0.67 

0.67 

Weapon  Envelope 

Outside 

-0.33 

-0.33 

Regional  Intelligence 

No  hostile  activities  known 

0.56 

0.44 

3.3  RELATIVE  IMPORTANCE  OF  CUES 

Participants  (N  =  8)  were  asked  to  rank  the  cues  in  relative  importance  to  the  assessment  of  threat. 
Origin,  Superstructure  Type,  and  Platform  Type  were  added  to  the  list.  There  were  1 7  items  to  rank. 
(Note:  Cargo  was  inadvertently  not  included.)  The  number  1  was  assigned  to  the  cue  with  the  highest 
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relative  importance.  The  number  17  was  assigned  to  the  cue  with  the  lowest  relative  importance. 
Data  from  eight  participants  were  tallied.  One  participant  was  dropped  because  of  outlying  values. 

Calculating  the  number  of  times  each  cue  was  assigned  to  a  particular  rank  (or  position),  and  then 
dividing  that  number  by  the  number  of  participants  computed  a  ranking  across  all  subjects  (the  mean 
proportion  that  each  cue  appeared  in  a  given  position).  For  example,  Table  3  shows  ESM  rankings. 
Two  participants  ranked  ESM  first  (Rank  =  1),  four  participants  ranked  it  third,  and  one  participant 
each  ranked  it  eighth  and  ninth.  Dividing  by  8  gave  the  mean  proportion  in  each  position  (Table  5). 


Table  5.  Example  of  rank  assignments  using  ESM. 


Rank 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Number  of  times  ESM  was 
assigned  to  a  given  Rank  by  the 
participants 

2 

0 

4 

0 

0 

0 

0 

1 

1 

Mean  Proportion 

0.25 

0.00 

0.50 

0.00 

0.00 

0.00 

0.00 

0.13 

0.13 

The  cue  with  the  highest  mean  proportion  in  each  position  was  the  most  representative  cue  at  that 
position.  Table  6  shows  the  rankings.  The  list  has  duplicate  entries  because  the  rankings  for  some 
cues  were  spread  across  many  ranks.  A  cue’s  position  in  the  list  determines  its  importance,  from  1 
(highest)  to  17  (lowest).  Table  7  lists  the  cues  without  duplicate  entries.  Duplicates  were  removed  by 
keeping  only  the  highest  ranked  position  for  any  cue. 


Table  6.  Relative  importance  of  cues  (with  duplicates). 


Rank 

Mean  Proportion 

Cue 

1 

0.25 

Platform/Wpn  Env/ESM 

2 

0.38 

Origin-Flag 

3 

0.50 

ESM 

4 

0.38 

Weapons  envelope 

5 

0.25 

Range 

6 

0.38 

Heading 

7 

0.25 

CPA/Speed 

8 

0.25 

CPA 

9 

0.25 

Number  of  vessels 

10 

0.25 

Own  support  in  area 

11 

0.25 

Destination 

12 

0.25 

Heading/History/Coordinated  ActivityA/oice  Communications 

13 

0.13 

CPA/Range/Number/Sea  Lane/Coordinated  ActivityA/oice 
communications/Own  support/Other  intelligence 

14 

0.25 

Superstructure/Voice  Communications/Destination 

15 

0.38 

Sea  lane 

16 

0.50 

Superstructure  type 

17 

0.38 

Other  intelligence 
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Table  7.  Relative  importance  of  cues  (without  duplicates). 


Rank 

1 

4 

5 

6 
7 

9 

10 
11 
12 
15 
17 


Cue 

Platform/Weapon  Envelope/ESM 

Origin-Flag 

Range 

Heading 

CPA/Speed 

Number  of  vessels 

Own  support  in  area 

Destination 

History/Coordinated  Activity/Voice  communication 
Sea  Lane/Other  intelligence 
Superstructure  Type 


3.4  INFORMATION  FLOW  IN  CIC 

Data  were  collected  on  information  flow  among  watch  standers  within  CIC.  Participants  were 
asked  to  list  their  most  recent  SUW  watch  station.  They  then  identified  other  watch  stations  that 
were  sending  and  receiving  information.  Figure  1  shows  the  flow  of  information  that  participants 
reported.  In  this  study,  participants’  most  recent  watch  station  experience  was  as  Antisurface 
Warfare  Coordinator  (ASUWC),  Antisubmarine  Tactical  Air  Controller  (ASTAC),  Antisubmarine 
Warfare  Coordinator  (SWC),  Electronic  Warfare  Supervisor  (EWS),  and  Combat  Systems  Coordi¬ 
nator  (CSC).  The  arrows  show  the  direction  of  communication  (one-  and  two-way).  The  dashed  lines 
mean  that  only  one  participant  mentioned  the  communication  channel.  Otherwise,  line  thickness 
represented  the  number  of  times  a  participant  mentioned  a  connection;  thicker  lines  mean  higher 
frequency. 
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MPA/HELO  CICWO 


Figure  1.  Surface  warfare  information  flow. 
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4.  SURFACE  THREAT  ALGORITHM 


Appendix  D  describes  a  preliminary  flowchart  for  a  rule-based,  surface  threat  algorithm.  The 
algorithm  is  based  on  the  air  threat  algorithm  that  was  developed  after  a  series  of  empirical  and 
observational  studies  of  U.S.  Navy  air  defense  personnel  (Liebhaber,  2001).  The  algorithm  incorpo¬ 
rates  and  analyzes  relevant  data  that  were  collected  for  this  study  and  computes  a  threat  rating  for 
each  surface  track. 
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5.  DISCUSSION 


Surface  Warfare  data  were  collected  from  highly  experienced  U.S.  Navy  personnel.  The  partici¬ 
pants  provided  baseline  threat  ratings  for  many  types  of  ships,  their  perception  of  threat  for  assess¬ 
ment  cues,  and  channels  of  information  flow  within  CIC.  Baseline  threat  ratings  were  consistent  with 
the  findings  from  air  defense  studies  (Liebhaber  and  Smith,  2000;  Liebhaber,  Kobus,  and  Smith, 
2000).  Participants  assigned  higher  threat  ratings  to  ships  in  a  littoral  environment  as  opposed  to 
ships  in  the  open  ocean.  Threat  change  ratings  showed  differences  in  each  operating  environment, 
but  the  effect  was  not  consistent  across  all  cues.  Relative  importance  of  cues  was  calculated  from 
participants’  ratings  of  threat  level  change  and  a  preliminary  surface  threat  algorithm  was  described. 
The  threat  and  threat  change  ratings  and  relative  importance  will  be  incorporated  into  future  devel¬ 
opment  of  the  SUW  threat  assessment  algorithm. 

While  this  study  describes  the  relationship  between  track  cues  and  changes  to  threat  rating,  that 
information  is  only  part  of  the  SUW  algorithm.  Work  is  needed  to  identify  how  SUW  personnel  use 
the  cues  to  evaluate  tracks  and  specify  the  relative  importance  of  the  cues.  These  data  will  be 
incorporated  into  the  algorithm,  enabling  it  to  better  capture  the  process  of  SUW  threat  assessment. 
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APPENDIX  A 


SURFACE  WARFARE  QUESTIONNAIRE 


The  format  of  the  questionnaire  has  been  slightly  modified  to  fit  within  this  report. 


Surface  Track  Analysis  Questionnaire _ 

Introduction 

•  The  purpose  of  this  questionnaire  is  to  gather  specific  information  about  the  data  that  you  use  when  you 
analyze  surface  contacts. 

•  Many  of  the  questions  are  about  single  pieces  of  infonnation  (e.g.,  speed,  sea  lane,  etc).  We  realize  that 
information  is  not  processed  in  isolation  in  the  real  situation.  However,  we  need  to  understand  how 
individual  elements  are  handled. 


Your  Background 

•  What  is  your  current  Rank/Rate?  _ 

•  Deployed  Surface  Warfare  Experience 

•  Please  indicate  the  number  of  years  or  months  you  spent  in  positions  where  you  were  directly  involved 
in  surface  craft  ID,  threat  analysis,  or  COI/CCOI  and  COA  determination. 

•  Answer  all  that  apply,  to  the  best  of  your  recollection. 


At-Sea  Experience 

SW  Position 

Years 

Months 

Commanding  Officer  (CO) 

Tactical  Action  Officer  (TAO) 

AntiSurface  Warfare  Coordinator  (ASuWC) 

Surface  Force  Track  Coordinator  (SFTC) 

Surface  ID  (SID) 

Tactical  Infonnation  Coordinator  (TIC) 

Auxiliary  Radar  Console  Operator  (ARC) 

Electronic  Warfare  Operator  (EWO) 

Electronic  Warfare  Supervisor  (EWS) 

Radar  System  Controller  (RSC) 

Other  SW  positions  not  listed  above 
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PART  1:  Threat  Posed  by  Different  Types  Of  Ships 


•  Based  on  your  experience,  how  threatening,  on  average,  is  this  type  of  ship? 

•  Please  circle  your  answer  for  each  environment. 


Origin/Flag/ID 

Friend 


Unknown,  Assumed  Friend 
Unknown  . 


Unknown,  Assumed  Enemy . 
Hostile  . 


“Littoral”  Environment 
Sometimes 

Never  Rarely  Often  Always 


“Open  Ocean”  Environment 
Sometimes 

Never  Rarely  Often  Always 


Superstructure 

Type  1  . 

Type  2  . 

Type  3  . 


Military  -  Potentially  Hostile  Vessels 


Carrier 


Submarine  . 

Surface  Combatant. 
Patrol/Escort . 


Amphibious  Warfare. 

Mine  Warfare . 

Special  Warfare/Ops. 

Intelligence  . 

Oiler  . 

Sealift  . 


Research 
Repair/Rescue . 
Service  Craft... 


Other  Auxiliary . 

Commercial/Private  Vessels 

Sealift  . 


Research 
Repair/Rescue . 
Service  Craft... 
Fishing 
Pleasure/Small 


PART  2:  Potentially  Threatening  Characteristics  or  Behaviors  Of  Ships 

•  Indicate  how  the  following  data  would  CHANGE  your  estimate  of  threat. 

•  Treat  each  item  as  if  were  the  only  piece  of  data  you  received  at  the  moment. 

•  Each  item  requires  two  answers  -  One  each  for  the  following  environments: 

o  Littoral 
o  Open  Ocean 

•  Please  answer  the  questions  as  accurately  as  possible. 

•  We  are  interested  in  your  at-sea  experience,  in  what  has  worked  for  you. 

Surface  ship  behaviors  -  How  would  the  values  below  CHANGE  your  estimate  of  threat? 


“Littoral”  Environment _ 

lower  lower  no  raise  raise 
Value  Ranges _  greatly  a  little  change  a  little  greatly 


Speed 

Speed  steady . 1  2  3 

Speed  increase . 1  2  3 

Speed  decrease  . 1  2  3 

Speed  of 

under  6  kts . 1  2  3 

6-10  kts . 1  2  3 

10-20  kts . 1  2  3 

over  20  kts . 1  2  3 

Course/Heading  relative  to  Own-Ship 

Steady  and  closing  . 1  2  3 

Steady  and  opening . 1  2  3 

Turn  to  closing . 1  2  3 

Turn  to  opening . 1  2  3 

CPA 

under  lnm . 1  2  3 

1-5  nm . 1  2  3 

5-15  nm . 1  2  3 

15-25  nm . 1  2  3 

25-50  nm . 1  2  3 

over  50  nm . 1  2  3 

Recent  maneuvers/history 

On  standard  patrol/route . 1  2  3 

Not  on  standard  patrol/route . 1  2  3 

Left  port  recently . 1  2  3 

Left  port  a  long  time  ago . 1  2  3 

Many  maneuvers/zig-zags . 1  2  3 

Changes  course  to  match 

own-ship  movements  . 1  2  3 

Unknown . 1  2  3 


4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 


5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 


“Open  Ocean”  Environment 

lower  lower  no  raise  raise 
greatly  a  little  change  a  little  greatly 


1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 
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“Littoral”  Environment _  “Open  Ocean”  Environment 


lower 

lower 

no 

raise 

raise 

lower 

lower 

no 

raise 

raise 

Value  Ranges  greatly 

a  little  change 

a  little  greatly 

greatly 

a  little  change 

a  little  greatly 

Range/Distance  from  Own-Ship 

Under  5  nm . 

2 

3 

4 

5 

i 

2 

3 

4 

5 

5-25  nm . 

2 

3 

4 

5 

1 

2 

3 

4 

5 

25-50  nm . 

2 

3 

4 

5 

i 

2 

3 

4 

5 

Over  50  nm . 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Within  Vital  Area . 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Within  CIE  Area . 

2 

3 

4 

5 

i 

2 

3 

4 

5 

Within  potential  weapon  range ... 

,.i 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Cargo 

Stacked  pallets . 

,..i 

2 

3 

4 

5 

i 

2 

3 

4 

5 

Arms/Ammunition . 

,.i 

2 

3 

4 

5 

i 

2 

3 

4 

5 

Refugees . 

,.i 

2 

3 

4 

5 

i 

2 

3 

4 

5 

Possible  contraband . 

...1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Number  of  vessels 

Unknown . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Single  contact . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

3-5  . 

,.i 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Over  5  . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Sea  Lane/T raffic  Lane 

On/following . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Near  . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Off  . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

ESM/Radar  Emitter 

No  emitter . 

..i 

2 

3 

4 

5 

i 

2 

3 

4 

5 

Emitter:  I-Band . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Emitter:  Non  1-Band . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Coordinated  Activity 

Train  ing/exercise . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Hiding  . 

Track  in  comm,  with  air,  surface, 

..1 

or 

2 

3 

4 

5 

1 

2 

3 

4 

5 

land  asset(s) . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Possibly  providing  targeting . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Voice  communication  with  track 

None  . 

,.i 

2 

3 

4 

5 

i 

2 

3 

4 

5 

Have  comm,  with  track . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Own  support  in  area 

None  available . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

Surface  available . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

DCA  or  Helo  available . 

..1 

2 

3 

4 

5 

1 

2 

3 

4 

5 
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“Littoral”  Environment _ 

lower  lower  no  raise  raise 
Value  Ranges  greatly  a  little  change  a  little  greatly 


Destination  of  track 

Headed  towards  standard  port . 1  2  3 

Not  headed  towards  std  port . 1  2  3 

Unknown . 1  2  3 

Weapon  Envelope  of  track  (Potential  or  Known) 

Outside  . 1  2  3 

Just  outside . 1  2  3 

Just  inside . 1  2  3 

Inside  . 1  2  3 

Regional  Intelligence 

Recent  activity  in/near  port . 1  2  3 

Military  exercises  in  area . 1  2  3 

No  military  exercises  known . 12  3 

No  hostile  activities  known . 1  2  3 

Anti  US  protests  in  region . 1  2  3 

Economic  summit  planned . 1  2  3 

Peace  summit  planned . 1  2  3 


4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 


5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 


“Open  Ocean”  Environment 

lower  lower  no  raise  raise 
greatly  a  little  change  a  little  greatly 


1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 
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Part  3:  Rank  data  in  order  of  relative  importance  to  determination  of  threat 

•  Begin  with  #1  as  MOST  important 

_ Origin/Flag 

_ Superstructure  Type 

_ Platform  Type 

_  Speed 

_____  Course/Heading  (in-/out-bound) 

_ CPA 

_ Recent  Maneuvers/History 

_ Range/Distance  from  Own-Ship 

_ Number  of  Vessels 

_ Sea  Lane/Traffic  Lane 

_ ESM/Radar  Emitter 

_ Coordinated  Activity 

_ Voice  communication  with  track 

_ Own  support  in  area 

_ Destination 

_ Weapons  envelope 

_ Other  intelligence 

Other: 


PART  4:  Surface  Warfare  Information  Flow 


•  Write  your  position  in  the  box  in  the  middle  of  the  page. 

•  On  the  lines  next  to  your  position,  list  the  positions  that  you  receive  information  from  and  send  information 
to. 


List  the  position(s)  that  you 

GET  information  from: 

a 


List  the  position(s)  that  you 
GIVE  information  to: 


r 


Your  most  recent  ASuW  position 


■=>« 


J 

THANK  YOU.  You  are  finished! 


V 
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APPENDIX  B 


SURFACE  WARFARE  CIC  WATCH  STATIONS 

Participants  indicated  that  they  had  experience  in  an  average  of  3.3  years  at  the  watch 
stations  described  in  Table  B-l. 


Table  B-1 .  Surface  warfare  CIC  Watch  Stations. 


Term 

Description 

ARC 

Auxiliary  Radar  Console  Operator 

ASTAC 

Antisubmarine  Tactical  Air  Coordinator:  Directs  operations  of  ASW  aircraft 

ASUWC 

Anti-Surface  Warfare  Coordinator:  Controls  the  surface  warfare  team,  directs 
and  integrates  track  assessment,  monitoring,  and  engagement 

CSC 

Combat  System  Coordinator:  Responsible  for  the  operation  of  the  AEGIS 
combat  systems 

EWO/EWS 

Electronic  Warfare  Operator/Supervisor:  Maintains  the  sensor-based 
electronic  emissions  and  warfare  situation. 

RSC 

Radar  System  Controller 

SID 

Surface  ID 

SFTC 

Surface  Force  Track  Coordinator 

SWS 

Surface  Watch  Supervisor 

TAO 

Tactical  Action  Officer:  Directs  and  coordinates  responses  to  threats  with  all 
warfare  coordinators 

TIC 

Tactical  Information  Coordinator:  Controls  sensor  operators  and  data  links 
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SURFACE  THREAT  ASSESSMENT  CUES 

Table  C-l  lists  surface  threat  assessment  cues  in  alphabetical  order. 


Table  C-1.  Surface  threat  assessment  cues. 


Cue 

Description 

Cargo 

Type  of  cargo  ship  is,  or  may  be,  carrying  (includes  people). 

Coordinated  activity 

Track  is  communicating  with,  or  nearby,  other  tracks  or  other  assets. 

CPA 

Closest  Point  of  Approach:  Estimated  distance  that  track  will  pass  by 
own  ship  if  the  track  and  own  ship  remain  on  their  current  courses. 

Destination 

Probable  destination  of  the  ship. 

ESM/Radar  emission 

Electronic  Support/Measures:  Electronic  emissions  from  the  track 
(typically  indicates  the  type  of  radar  system  the  track  is  using). 

Heading 

Direction  ship  is  traveling  relative  to  own  ship  (i.e.,  opening  or  closing). 

Number  of  vessels 

Number  of  ships  in  close  proximity. 

Own  Support 

Availability  of  nearby  friendly  ships  or  patrol  aircraft  (DCA  or  HELO) 

Range/Distance 

The  track’s  distance  from  own  ship. 

Recent  history 

The  track’s  recent  maneuvers. 

Regional  Intelligence 

Geopolitical  and  military  situation  in  the  region  the  ship  is  operating. 

Sea  Lane 

A  published,  typical,  or  otherwise  known  route  of  travel. 

Speed 

Approximate  speed  or  an  indication  of  change  (e.g.,  increasing). 

Voice  communication 
with  track 

Indicates  if  ownship,  or  nearby  friendly  asset,  has  communicated  with 
the  track. 

Weapon  envelope 

The  track’s  position  with  respect  to  its  estimated  weapons  envelope 
(e.g.,  within). 
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APPENDIX  D 


SURFACE  THREAT  ALGORITHM  SPECIFICATIONS 


A  preliminary  rule-based,  surface  threat  algorithm  is  described  below.  It  is  based  on  the  air  threat 
algorithm  that  was  developed  after  a  series  of  empirical  and  observational  studies  of  U.S.  Navy  air 
defense  personnel  (Liebhaber,  2001).  In  its  current  form,  the  algorithm  incorporates  and  analyzes 
relevant  data  (as  identified  by  U.S.  Navy  surface  warfare  experts)  and  computes  a  threat  rating  for 
each  surface  track. 

SURFACE  THREAT  ALGORITHM  DETAIL 

Processing  is  divided  into  three  stages.  Baseline  threat  level  is  computed  in  stage  one  from  ID  and 
Platform  Threat  Ratings  (see  Table  1).  Threat  assessment  is  then  conducted  by  evaluating  each  of  the 
cues  in  turn,  and  adjusting  threat  level  accordingly.  Threat  level  is  adjusted  by  adding  or  subtracting 
Threat  Change  Ratings  from  Table  2. 

Critical,  or  high  weight,  cues  are  evaluated  first;  in  stage  two.  Critical  cues  are  those  the  cues  most 
often  used  by  experienced  ASuW  personnel.  The  best  guide  at  this  time  is  the  indication  of  relative 
importance  in  Table  5.  Remember  that  importance  is  approximated  by  list  position.  Cues  with  the 
highest  weight  are  the  critical  cues,  however,  the  high/low  cutoff  value  must  be  empirically 
determined.  In  stage  three,  the  remaining  cues  are  evaluated  only  if  exceptions  are  encountered  in 
one  or  more  of  the  critical  cues.  Exceptions  are  data  values  that  are  unexpected  or  atypical  (e.g., 
increasing  speed  for  a  nearby  small  ship).  Their  values  must  also  be  empirically  determined. 
Exception  Score  is  used  to  determine  whether  or  not  to  evaluate  additional  cues  beyond  the  critical 
cues.  It  is  initialized  to  zero.  When  the  algorithm  processes  a  cue  value  that  is  an  exception,  the 
weight  of  that  cue  is  subtracted  from  Exception  Score.  If  the  value  of  Exception  Score  is  less  than 
zero  after  all  of  the  critical  cues  have  been  processed,  then  the  algorithm  will  process  additional 
cues.  The  weights  of  the  additional  cues  are  added  to  Exception  Score  (unless  they  were  an 
exception,  too).  The  assumption  is  that  an  unexpected  value  for  a  given  cue  implies  a  loss  of 
information.  That  loss  is  captured  in  the  algorithm  by  subtracting  the  weight  of  that  cue  from  the 
Exception  Score.  Additional  cues  must  be  processed  until  that  loss  is  made  up. 

Stage  1.  Compute  baseline  threat  level 

la.  Baseline  Threat  Rating  =  ID  Threat  Rating  +  Platform  Threat  Rating 

•  Example:  Baseline  Threat  Level  of  an  Unknown  (2.89)  Military  Patrol/Escort  (3.67) 
ship  is  2.89  +  3.67  =  6.56 

Stage  2.  Adjust  threat  rating  using  the  critical  track  cues 

For  each  critical  cue  do 

2a.  Get  data  value  for  the  current  cue 

2b.  New  Threat  Rating  =  Old  Threat  Rating  ±  Threat  Change  Rating  for  current 
data  value 
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•  Note:  When  the  first  cue  is  evaluated,  Old  Threat  Rating  equals  the  Baseline 
Threat  Rating. 

•  Example:  Threat  Rating  of  an  Unknown  Patrol/Escort  (6.56)  that  is  slowing 
down  in  a  littoral  environment  (0.44)  is  6.56  +  0.44  =  7.00. 

2c.  IF  Value  for  current  cue  is  an  exception 

THEN  New  Exception  Score  =  Old  Exception  Score  -  Weight  for  current  cue 

Stage  3.  Adjust  threat  rating  using  additional  cues,  if  necessary 

While  Exception  Score  <  0  do 

3a.  Get  data  value  for  the  current  cue 

3b.  New  Threat  Rating  =  Old  Threat  Rating  ±  Threat  Change  Rating  for  current  data 
value 

3c.  New  Exception  Score  =  Old  Exception  Score  +  Weight  for  current  cue 
3d.  IF  Value  for  current  cue  is  an  exception 

THEN  New  Exception  Score  =  Old  Exception  Score  -  Weight  for  current  cue 
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